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Abstract 

The chart of Indonesian student of mathematical ability development in 
Program for International Student Assessment (PISA) event during the 
last 4 periods shows an unstable movement. PISA aims to examine the 
ability of children aged 15 years in reading literacy, mathematics 
literacy, and science literacy. The concept of mathematical literacy is 
closely related to several other concepts discussed in mathematics 
education. The most important is mathematical modelling and its 
component processes. Therefore the goal of this research is to 
investigate secondary school students’ difficulties in modeling 
problems PISA-model level 5 and 6. Qualitative research was used as 
an appropriate mean to achieve this research goal. This type of research 
is a greater emphasizing on holistic description, and phenomenon 
identified to be studied, students’ difficulties in modelling real world 
problem in PISA model question level 5 and 6. 26 grade 9 students of 
SMPN 1 Palembang, 26 grade 9 students of SMPK Frater Xaverius 1 
Palembang, and 31 participants of mathematical literacy context event, 
were involved in this research. The result of investigate showed that 
student is difficult to; (1) formulating situations mathematically, Such 
as to representing a situation mathematically, recognizing mathematical 
structure (including regularities, relationships, and patterns) in 
problems, (2) evaluating the reasonableness of a mathematical solution 
in the context of a real-world problem. The students have no problem in 
solve mathematical problem they have constructed. 

Keywords: Mathematical model, Modelling competence, PISA, PISA 
Questions level 5 and 6, Students’difficulties in solving 
PISA-model Questions. Mathematics Literacy. 


Abstrak 

Grafik perkembangan kemampuan matematika siswa Indonesia pada 
ajang PISA selama 4 periode PISA terakhir menunjukkan pergerakkan 
yang tidak stabil. PISA bertujuan meneliti secara berkala tentang 
kemampuan anak umur 15 tahun dalam membaca (reading literacy), 
matematika (matehematics literacy ), dan IPA (science literacy), konsep 
literasi berkaitan erat dengan beberapa konsep-konsep lain yang 
dibahas dalam pendidikan matematika, tetapi yang paling penting 
adalah modelling (pemodelan matematika) dan komponen prosesnya. 
Oleh karena itu tujuan dari penelitian ini adalah untuk mengetahui 
kesulitan siswa sekolah menengah pertama memodelkan masalah 
model PISA level 5 dan 6. Penelitian yang digunakan untuk mencapai 
tujuan penelitian ini adalah penelitian kualitatif. Jenis penelitian ini 
adalah lebih menekankan pada deskripsi secara keseluruhan, dan 
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mengidentifikasikan kesulitan siswa dalam membuat model 
matematika dari soal PISA level 5 dan 6. 26 orang siswa kelas 9 SMPN 
1 Palembang, 26 orang siswa kelas 9 SMPK Frater Xaverius dari 1 
Palembang, dan 31 peserta kontes literasi matematika di Palembang, 
terlibat dalam penelitian ini. Hasil investigasi menunjukkan bahwa 
siswa mengalami kesulitan dalam proses; (1) merumuskan masalah 
dalam kehidupan sehari-hari ke dalam model matematika, Seperti 
memginterpretasikan konteks situasi nyata ke dalam model matematika, 
memahami struktur matematika (termasuk keteraturan, hubungan, dan 
pola) dalam masalah, (2) mengevaluasi kewajaran dari solusi 
matematika dalam konteks masalah dunia nyata. Namun siswa tidak 
memiliki masalah dalam menyelesaikan model matematika yang telah 
mereka bangun. 

Kata Kunci: Model matematika. kompetensi pemodelan matematika, 
PISA, Soal model PISA level 5 dan 6. Kesulitan siswa 
menyelesaikan soal PISA. Literasi matematika. 


Introduction 

Program for International Student Assessment (PISA) is conducted by the OECD 
(Organization for Economic Co-operation & Development) and United Nations 
Educational, Scientific and Cultural Organization (UNESCO) Institute for Statistics. 
PISA aims to examine the ability of children aged 15 years in the regular reading 
(reading literacy), mathematics (mathematics literacy), and Science (science literacy). 
Indonesia is one of the PISA participating countries that have joined since 2000. The 
chart of Indonesian student of mathematical ability development in the PISA event 
during the last 4 periods shows an unstable movement, Indonesian’s students only 
able to answer questions PISA level 1, 2 and 3, and a few students can solve level 4 
questions. Chairman of an international group of mathematicians for PISA 2012, 
Kaye Stacey (2010), argued that the concept of literacy is closely related to several 
concepts discussed in mathematics education. But most important is the modeling 
because the cycle of mathematical modeling is a central aspect of the conceptions of 
PISA students as an active problem solvers, but students or problem solver often do 
not need to be involved in every stage of the cycle of modeling, especially in the 
assessment context, Blum, Galbraith, Henn & Niss, (2007). Therefore, researchers 
interested in conducting research to investigate middle school students’ difficulties in 
modeling PISA model problems level 5 and 6. 
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Theoretical framework 

The structure of the PISA mathematics framework can be characterised by the 
mathematical representation: ML + 3Cs. ML stands for mathematical literacy, and the 
three Cs stand for content, contexts and competencies. The PISA mathematical 
literacy domain is concerned with the capacities of students to analyse, reason, and 
communicate ideas effectively as they pose, formulate, solve and interpret 
mathematical problems in a variety of situations. In using the term “literacy”, the 
PISA focus is on the sum total of mathematical knowledge a 15-year-old is capable of 
putting into functional use in a variety and some creativity. 

1. Mathematical Literacy 

The definition of mathematical literacy refers to an individual’s capacity to formulate, 
employ, and interpret mathematics, and this language provides a useful and 
meaningful structure for organising the mathematical processes that describe what 
individuals do to connect the context of a problem with the mathematics and thus 
solve the problem. The categories to be used for reporting are as follows: 

Formulating situations mathematically 

The word formulate in the mathematical literacy definition refers to individuals being 
able to recognize and identify opportunities to use mathematics and then provide 
mathematical structure to a problem presented in some contextualized form. In the 
process of formulating situations mathematically, individuals determine where they 
can extract the essential mathematics to analyze, set up, and solve the problem. They 
translate from a real-world setting to the domain of mathematics and provide the real- 
world problem with mathematical structure, representations, and specificity. They 
reason about and make sense of constraints and assumptions in the problem. 

Employing mathematical concepts, facts, procedures and reasoning 
The word employ in the mathematical literacy definition refers to individuals being 
able to apply mathematical concepts, facts, procedures, and reasoning to solve 
mathematically-formulated problems to obtain mathematical conclusions. In the 
process of employing mathematical concepts, facts, procedures and reasoning to solve 
problems, individuals perform the mathematical procedures needed to derive results 
and find a mathematical solution (e.g., performing arithmetic computations, solving 
equations, making logical deductions from mathematical assumptions, performing 
symbolic manipulations, extracting mathematical information from tables and graphs, 
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representing and manipulating shapes in space, and analyzing data). They work on a 
model of the problem situation, establish regularities, identify connections between 
mathematical entities, and create mathematical arguments. 

Interpreting, applying and evaluating mathematical outcomes 
The word interpreting used in the mathematical literacy definition focuses on the 
abilities of individuals to reflect upon mathematical solutions, results, or conclusions 
and interpret them in the context of real-life problems. This involves translating 
mathematical solutions or reasoning back into the context of a problem and 
determining whether the results are reasonable and make sense in the context of the 
problem. This mathematical process category encompasses both the “interpret” and 
“evaluate” arrows noted in the previously defined model of mathematical literacy in 
practice. Individuals engaged in this process may be called upon to construct and 
communicate explanations and arguments in the context of the problem, reflecting on 
both the modeling process and its results. 

2. Fundamental Mathematical Capabilities Underlying the Mathematical 
Processes 

A decade of experience in developing PISA items and analyzing the ways in which 
students respond to items has revealed that there is a set of fundamental mathematical 
capabilities that underpins each of these reported processes and that also underpins 
mathematical literacy in practice the seven fundamental mathematical capabilities 
used in this framework are as follows: 

Communication, Mathematising, Representation, Reasoning and argument, Devising 
strategies for solving problems, Using mathematical tools. 

3. Context 

The reference to “a variety of contexts” in the definition of mathematical literacy is 
purposeful and intended as a way to link to the specific contexts that are described and 
exemplified more fully later in this framework. The specific contexts themselves are 
not so important, but the four categories selected for use here (personal, occupational, 
societal, and scientific) do reflect a wide range of situations in which individuals may 
meet mathematical opportunities. 

4. Mathematical Content 

PISA aims to assess students’ capacity to solve real problems, and therefore includes 
a range of mathematical content that is structured around different phenomena 
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describing mathematical concepts, structures or ideas. This means describing 
mathematical content in relation to the phenomena and the kinds of problems for 
which it was created. In PISA these phenomena are called overarching ideas; (1) 
Change and relationships, PISA recognises the importance of the understanding of 
change and relationships in mathematical literacy. (2) Space and shape, PISA 
recognises that patterns are encountered not only in processes of change and 
relationships, but also can be explored in a static situation. (3). Quantity, PISA 
recognises the importance of quantitative literacy. In PISA, the overarching idea of 
quantity includes: meaning of operations, feel for magnitude of numbers, smart 
computations, mental arithmetic, estimations. (4) Uncertainty, In PISA the 
overarching idea of uncertainty is used to suggest two related topics: statistics and 
probability. Both of these are phenomena that are the subject of mathematical study. 

5. Proficiency Levels of modelling competence 

Proficiency levels for modeling, the following development is observed as literacy 
levels increase. 

1. Apply simple given models 

2. Recognize, apply and interpret basic given models 

3. Make use of different representational model 

4. Work with explicit models, and related constraints and asumtion 

5. Develop and work with complex models; reflect on modelling processes and 
outcomes 

6. Conceptualize and work with models of complex mathematical procesess and 
relationships reflect on, generalize and explain modeling outcomes 

The process of modeling constitutes the bridge between mathematics as a set of tools 
for describing aspects of the real world, on the one hand, and mathematics as the 
analysis of abstract structures, on the other; as such it is a pervasive aspect of 
mathematics. Figure 1 schematically represents this mathematical modeling approach 
by Erik DE CORTE 
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Figure 1. Schematic diagram of the process of modeling 


Method 

The research methodology that we use in this study is a qualitative research. 
Qualitative research is research studies that investigate the quality of relationships, 
activities, situations, or materials. This type of research is a greater emphasize on 
holistic description—that is, on describing in detail all of what goes on in a particular 
activity or situation rather than on comparing the effect of a particular treatment or on 
describing the attitudes or behaviors of people. There are five steps in qualitative 
research (Fraenkel & Wallen, 2010): (1) identification of the phenomenon to be 
studied - Students difficulty in construct mathematical model of real world problems 
in PISA- model questions; (2) identification of the participants in the study - 26 third 
grade secondary school students aged 15 years old in SMPN 1 Palembang, 26 third 
grade secondary school students aged 15 years old in SMPN 1 Palembang SMPK 
Frater Xaverius 1 Palembang, 31 participants of mathematical literacy context event; 
(3) generation of hypotheses - students can doing well in formulating real world 
problem into mathematical problem, employ mathematical problem, and interpret 
mathematical solution to real world situation; (4) data collection - used students 
worksheet, video recording and interviewed some students to get deeper information 
of their thinking process; data analysis - data will be analyzed by holistic descriptive; 
and (5) interpretation and conclusion - using indicator of modelling competence, 
mathematical literacy refer to proficiency level of PISA question given as a guideline 
to interpret and to make a conclusion. 
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Result and Analysis 

Students were asked to solve two math PISA model to measure the students’ ability of 
modeling. Students are also given a paper to write a commentary on the questions 


provided. After students complete the given problem, they also interviewed to clarify 
their answers, and asked the cause of their difficulties in solving problems. The 


following are two models of the PISA math level 5 and 6 are assigned to students. 


Question 1 


Question 2. 




How many cube, cuboids, Cylinder that may be 
added at the right side of third balance such that 
the weights are in balance? Give 2 the 
combination of the types of things are possible. 


Ira and ATI each have candies. 
Here is a conversation between 
Ira and Ati 

Ira: Ati, will you give me 5 
candies of yours so my 
candies as much with your 
own candies. 

Ati: (Laughter) No Ira. Why do 
not you give me 5 candies 
so my candies as much 
twice of yours candies. 

Question 

Problem 11 How many candy 
wholly owned by Ira and Ati? 
Write down how the 
calculations. 


Context : Candies 

Conten : Change and 

relationships 

Competence : Kompetensi 

pemodelan 


Context 

Conten 

Competence 

Level 


: Balance 

: Change and relationships 
: modelling 
: 5 


Mathematics Literacy : 

Formulating situations mathematically 
Employing mathematical concepts, facts, 
procedures, and reasoning 
Interpreting, applying and evaluating 
mathematical outcomes 


Level : 6 
Mathematics literacy: 

Formulating situations 
mathematically 
Employing mathematical 
concepts, facts, procedures, 
and reasoning 

Interpreting, applying and 
evaluating mathematical 


outcomes 
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Students' answers to First question 

Students answers will be analyze to get the information of student’s difficulty in 
solving the PISA question. 


Student A 



Students' work shows that students write the weight of cylinder, cube and cuboids 
directly without any explanation. However, student gave the correct answers and 
logic thinking to get the number of objects that must be added to the third weights 
such that the third weight. But the students do not provide an explanation or any kind 
of work steps that make readers understand his thinking process. Student’s 
explanation of how he determine the number of object that should be add in the right 
side of third weights such that the third weights is balance, shows that students 
understand the problems. He said that he calculated the balance weight of the object 
on the left side of the third weights based on the weight of each object that he already 
knew in prior step. Then trying to determine what objects should be added to the 
third-right weights, and what is the weight of the three objects in order to balance 
weights. Therefore, the modeling process which appears only process Interpreting, 
applying and Evaluating mathematical outcomes, while the process formulate 
problems into mathematical models and solve mathematical models are not visible 
from the students' responses. 
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As a follow-up action to clarify the students' answers, the students concerned were 
interviewed with the conversation as follows: 


Researcher 

Student 

Researcher 

Student 

Researcher 

Student 


Researcher 


Student 


How do you determine the weight of cubes, cylinder and cuboids? 
By using my logic 

Can you tell me the process to get the answer? 

By comparing the object is reduced on one side of the scales with 
the object increases on another side 
So how do the comparison? 

There are 3 cubes on the left side of weights , whereas on the 
right side of a cube is reduced while the number the cuboids 
increasing 1 unit and so does cylinder, so the weight of a cube is 
equal to 1 cuboids + 1 cylinder. Similarly, the second balance. The 
cube is reduced lunit while cuboids increased 3 unit from the left 
side to right side of second weights. Thus a cube is equal to 3 
cuboids. Therefore a cube weighs 3 kg. Due to heavy weight a 
cube equal to the weight of a cuboids plus the weight of a cylinder. 
Cuboids’ weight is 1 kg, so the weight of cylinder is 2 kg 
Excellent, why do not you write down how you get that answer, so 
people who check your answer sure that you understand about 
that? 

I find it hard to write regularly, I just think in head and count it 
directly. 


In addition, other students also have similar answer with student A. 


Student B 

, cW v 


Fig. 1. The answer of student B 

Almost the same as the previous student, this student also does not provide any 
explanation. In fact none of the modeling process that emerged from students' 
responses. Here is the result of conversations with students in the interview process. 


Researcher 

Students 


Researcher 

Student 


How did you get answers like this? 

To get the first combination, the numbers of objects on the right side 
of the third weights are equated with the number of objects on the left 
side of the third weights. 

What about the second combination? 

By comparing objects on one side of the scales decreases with 
increasing object on the other side (the same thought process to 
previous student, but students 2 makes mistake in calculating the 
number of cylinder that grow on the right side of weights ). 

On the left side of weights first, there are 3 cube, whereas on the 
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right side of a cube is reduced while increasing the number of 
cuboids and cylinder increase 1 and 2 (so he gave the wrong answer 
Researcher : Why do not you write down how you get that answer, so people who 
check your answer sure that you understand about that? 

Student : I do not know how to write it. 

Students C 



Fig.3. The answer of Student C 


Student C was able to simplify the problem into symbols and mathematical language 
correctly, and give the correct answer. Modeling process that appears in her answer 
are formulate the problem into a mathematical model, and interpret the problem, but 
the process of solving a mathematical model has been formulated not appear. From 
interviews in mind that students are hard to write how to get results. Students' 
responses are almost the same as the student answers D. 


Student D 

O - 30? 

0 ^ ^ 


VorofovirO-^V I 

3 - ^ 0, M Q 


1 (<Eyrr\o 3N 2. 

\ y O’ © j q cfzi? 


Fig.4. The answer of Student D 


Visibly, student C and student D’s way in modeling is almost same. Nevertheless, 
they have different modeling process. Students C modeling the problem by directly 
writing the number of objects that exist at weights 1 and 2, while the student D write 
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the model by compare the number of objects decreases and increases on the sides of 
the scales first. 

Student E 



s> ^ 7-x * + 2t = •»<-*• -+ 3% 

- >c 

krtoOTC) z ^ 

* X - ^ = O 

o A. 


A ^ - A*x rA'b 

* - 

V = ?>% 

^ ~ O ^ ■ *t - 

i^=a. 

^ ^ ^ 3> ~L^L 

W = * ** 4 





Fig.5. The answer of Student E 


Different from the four previous students. Student E can construct mathematical 
model through the correct process. The teacher of student E said that student E is the 
student who has high achievement in mathematics. 

In contrast, the low achievement student cannot do more than write comment for the 
questions. Those are their comments; 




♦Drain brain 


CjjVV Va^V V<cxWcv>\cv^ c 

\oq\^c<A 


—►difficult to calculate without 
using a calculator 

> Hard to find a suitable formula 
to solve the problem 

> Using the deeper logic 


Fig.6. students’ comment on first question 
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Students Answer for The Second Question 
Student F 

Soot\ (i 

«V\ S' fv\^rv 


2. ^crnfvfin tcq 


p?rrA£A CXt\ A- S'ocf'TV^ - 
Ocrr^ \\c\ -1 l S^>r»A.ef' -t T'^crr^w ~ 
(Oj mn^ - |p?tiv\tr\ h~A 

^tnuiA CAf( - irt\ -+ S oeerr^N 

r (0 4 S’ - 

- \-STpf r rr ^4 

Fig.7. The answer of Student F 


rx VfCA 
- 2rx>vr^ ,<0 


Students simplify the problem and construct a mathematical model. But students are 
not rigorous thinking and do not fully understand the problem. He did not realize that 
Ati’s candies reduced when Ira’s Candies increase and Ati’s candies increase when 
Ira’s Candies reduced. She also solve the model has formulated, but the student gives 
the wrong answer, because the model he built is wrong. While the process of 
interpreting and evaluating the modeling does not appear from the answers above. 


Student G 



1 1 Ira - kp 

At\ - 5 -KW • 

4 

Ip-f 5If x r^r-w 


6ot — (o —v- 2-V 


425 *= ID 


'Vt - -^2- 


Ira- 2/!5 per(/pn -X—* 

Ati - T,? 



Candies 



Fig. 8. The answer of Student G 
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Students are seen trying to build a mathematical model, but he was wrong in 
formulate the relationship of the number of Ati and Ira’s Candies. Students are also 
tried to solve model that has been built which is definitely wrong, because he has built 
a wrong model. Problems that form the concern here is process of evaluating the 
suitability of student mathematical solution which he acquired with the fact that it is 
impossible ira has 2.5 candy and candy Ati has 7.5. 


Students H 


fp^ 'rf» *i*r> : O 

iorr 5 h t IjpTi Vs ■ v’"’ ■ 


The number of Ira’s Candies: 35 
Number of Ati’s candies : 25 

I calculate it based on conversations and use slightly instinct. If 5 of Ira’s candies 
given to Ati, so Ira candies is 30 pieces and Ati’s candies is 30 pieces, if 5 of Ati’s 
candies given to Ira, So Ira’s candies is 40 and Ati’s candies is 20. 


Fig.9. The answer of Student H 

Student H was able to answer this problem correctly. In the interview he said that he 
solve the problem by simply using instinct, with the thought process as stated in the 
answer. 


Student I 

U 


ho AC 


as~l ,5 

0c> x Uo (ff KjyO 


S5 **^35 

'io = 3o 

3ad\ , jomVA, X ra d*. Aft 60 U. 


Therefore, The total number of Ira’s and Ati’s candies is 60 piecess. 


Fig.9. The answer of Student I 
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Based on students' answers on the answer sheet and the results of interviews, it is 
known that he is not using the process of modeling first. But he tried to choose any 
number that fulfill to number of Ira and Atis’ candies according to the conversation. 
Then prove or evaluate whether or not in accordance with the Ari and Ati’s 
conversation. 

Student E 

Soa\ J 

K - y 

x-v 

Fig. 10. The answer of Student I 

Student E is the high achievement students who can solve first question in the correct 
modeling process. Otherwise, he formulated this real world problem into incorrect 
mathematical problem and then he cannot solve the problem and interpret the solution 
to real situation. The student said in the interview process that she hard to construct 
mathematical model of this second problem because she not accustom to solve this 
type of problem. Whereas, the first problem is similar with the problem that she had 
learn in system of linear equations of two variables. 

Meanwhile, The comments of students who cannot solve this question are as follows: 

► Confuse 

_ Hard to find the formula that appropriate to 

vCfnOi ► 11 1 

solve the problem 
.&cry hard 

Fig. 11. The answer of Student F 

Same with student comments on the previous problem, the students see the biggest 
difficulty in formulating a problem in everyday life into a mathematical model. 


7 . 

>0\\A 
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Conclusion 

Of the various students' answers, both of Question 1 and Question 2 can be concluded 
The result of investigate showed that student is difficult to; (1) formulating situations 
mathematically, Such as to representing a situation mathematically, recognizing 
mathematical structure (including regularities, relationships, and patterns) in 
problems, (2) evaluating the reasonableness of a mathematical solution in the context 
of a real-world problem. The students have no problem in solve mathematical 
problem they have constructed. The founded in this research is most of high 
achievement students cannot solve uncommon problem completely since they cannot 
formulate the problem mathematically, while students with moderate achievement can 
solve the problem by using their own way that they called “insting”, “trial and error”, 
“using own logic”, whereas the low achievement students cannot solve the two 
problem because they cannot find appropriate formula that they can used to solve the 
problem, and the problem required high logic. 
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